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Abstract  This  paper  describes  an  approach  to  parallel  object-oriented 
simulation.  Parallel  evaluation  of  simulation  programs  is  accomplished  by 
compiling  objects  into  sets  and  production  rules  so  that  they  can  be  evaluated 
with  parallel,  set -oriented  operations  which  effectively  utilize  the  capacity  of 
parallel  processors  with  minimal  communication  overhead 


1.  Introduction 

Since  Smalltalk  was  introduced  in  the  early  80's,  it  has  been  generally 
accepted  that  object-oriented  programming  languages  can  provide  a  number 
of  attractive  features  for  software  development  In  an  object-oriented  system, 
data  can  be  naturally  expressed  as  a  set  of  t  bjects.  This  feature  is  very  general 
and  can  handle  any  type  of  conceptual  abstraction  and  organization.  A 
collection  of  abstract  classes  and  their  ^Siooated  algorithms  (methods), 
constitute  the  kind  of  framework  into  which  particular  applications  can  insert 
their  own  specialized  code  by  constructing  concrete  subclasses  that  work 
together.  With  the  method /message  protocol,  objects  can  be  accessed 
without  knowledge  of  their  internal  structures.  Classes  can  be  organized  into 
hierarchies  to  that  all  methods  and  attributes  erf  a  class  can  be  automatically 
inherited  by  all  of  its  subclasses.  In  addition,  the  nature  of  object-oriented 
representations  suggests  that  an  object-oriented  simulation  program  can  be 
executed  in  parallel. 

Unfortunately,  despite  the  nice  features  described  above,  the  object- 
oriented  methodology  has  focused  on  general-purpose  programming  with  no 
definition  of  object  control,  management,  and  information  retrieval.  Our 
research  by  and  large  addresses  the  area  called  "active  object  base".  Our 
definition  of  active  objects  goes  beyond  the  traditional  object -on  en  led 
paradigm  by  including  a  control  component  in  each  object,  so  that  it  can 
always  be  active,  capable  of  performing  state  transitions  and  handling 
asynchronous  events.  Hence  an  active  object  base  is  defined  to  be  a  large 
collection  of  objects  instrumented  with  states  and  asynchronous  state 
transitions.  Such  a  model  can  be  best  illustrated  by  a  battlefield,  in  which  all 
command  and  control  units  are  active  objects,  and  whose  states  are  extremely 
sensitive  to  environmental  changes.  Our  research  addresses  five  major 
problems  associated  with  active  object  bases:  representation,  presentation, 
management,  programming,  and  execution.  This  paper  reports  the 
representation  and  execution  aspects  of  an  object  base,  taking  simulation  as 
the  domain.  By  representing  the  control  portion  of  an  active  object  as  a 
production  system,  parallel  evaluation  of  a  simulation  program  is 
accomplished  by  compiling  objects  onto  an  interconnected  network,  so  that 
they  can  be  evaluated  with  parallel,  set-oriented  operations  which  effectively 
utilize  the  capacity  of  parallel  processors  with  minimal  communication 
overhead.  This  paper  is  organized  into  the  following  sections:  Survey  of 
Related  Work#  The  Object  Model,  Object-Oriented  Simulation,  Rule 
Processing  Incremental  Network  Compilation.  Parallel  Evaluation,  Primitive 
Experiment  Results,  and  Conclusion. 


can  be  found  in  [ThMU90)  Typically,  such  systems/languages  extend  an 
object-oriented  programming  language  with  the  necessary  constructs  for 
simulation,  for  Instance,  with  ‘...the  notion  of  simulation  time  and 
mechanisms  for  entities  in  the  language  to  manipulate  simulation  time  " 
[Mors90] 

The  problem  with  parallel  processing  of  simulation  systems  ha:  at¬ 
tracted  much  attention  recently.  A  number  of  parallel  computation  models 
and  their  associated  problems  have  been  investigated  {FupWJfMors^O}  The 
models  can  be  classified  into  two  categories  synchronous  and  asynchronous 
In  a  synchronous,  parallel  simulation  system,  proces ses  and  even's  are 
scheduled  and  executed  by  the  simulator  with  a  global  clock  Each  process, 
however,  in  an  asynchronous,  parallel  simulation  system  maintains  us  own 
dock,  and  processes  and  events  are  scheduled  and  executed  in  a  fully  dis 
tributed  fashion.  This  means  there  exists  no  scheduler  to  synchronize  the 
events  globally.  According  to  fFuji901,  "  .  few  simulator  events  occur  at  any 
single  point  in  simulated  time;  therefore  parallelization  techniques  based  on 
lock-step  execution  using  a  global  simulation  dock  perform  poorly  or  require 
assumptions  in  the  timing  model  that  may  compromise  the  fidelity  of  the 
simulation-"  Accordingly,  "Concurrent  execution  of  events  at  different  points 
in  simulated  time  is  required,  but.  ..  this  introduces  interesting  synchroniza¬ 
tion  problems..."  Most  of  these  synchronization  problems  arise  from  data 
dependencies  among  different  processes  which  run  at  different  speeds 
Approaches  to  the  synchronization  problems  can  be  in  turn  distinguished  by 
two  approaches:  conservative  and  optimistic.  A  conservative  approach 
prevents  any  synchronization  problem  from  happening,  but  it  degrades  per¬ 
formance.  An  optimistic  approach  allows  synchronization  problems  to  oc¬ 
cur,  and  rollbacks  are  often  necessary  once  these  problems  are  detected 

QhiakQnaitaL  Dsl tatasa 

In  the  past,  several  object-oriented  databases  have  been  proposed  In 
brief,  researchers  and  developers  have  approached  object-oriented  database 
implementation  along  two  lines:  by  extending  the  relational  model  (e  g  . 
P05TGRES  [5ton86J  IS1R086]  (Ro6t87),  GENESIS  (BBGS861.  Starburst 
I5CFL06],  Iris  {Fish 87),  EXODUS  (GDRS86L  and  PROBE  (DBCH86)),  or  by 
applying  the  ideas  of  object-oriented  programming  to  permanent  storage 
(e.g,  GemStone  JMSCP86)  and  Jasmine  (MaYViBoj  (Wieb86J>  Meet  of  the 
systems  in  the  first  category  have  been  designed  to  simulate  semantic  data 
models  by  including  mechanisms  such  as  abstract  data  types,  procedural 
attributes,  inheritance,  union  type  attributes,  and  shared  subobjects  And 
most  of  the  systems  in  the  second  category  extend  an  object-oriented 
programming  Language  with  persistent  objects  and  some  degree  of 
declarative  object  retrieval 

Both  approaches  have  drawbacks  in  processing  a  large  number  of 
active  objects.  The  first  approach  suffers  from  the  instability  problem 
resulting  from  the  separation  of  control  and  data  The  second  approach,  on 
the  other  hand,  loses  the  advantages  provided  by  fact-oriented  database 
operations. 


2-  Survey  of  Related  Work 

Work  related  to  the  simulation  environment  described  in  this  paper  can 
be  classified  into  three  categories;  object-oriented  and  parallel  simulation 
systems,  object-oriented  databases,  and  active  databases. 

Object-Oriented  and  Parallel  Simulation  Systems 

The  features  provided  by  the  object-oriented  paradigm  naturally  lead 
to  the  use  of  the  object-oriented  paradigm  in  simulation  systems  To  our 
knowledge,  more  than  a  do2en  object-oriented  simulation  lan¬ 
guages/systems  have  been  developed;  a  survey  of  such  systems/languages 


This  work  is  funded  by  the  Office  of  Naval  Technology,  Code  227,  Computer 
Technology  Block  Program  through  the  Office  of  Naval  Research  ASF.F.  Summer 
Faculty  Research  Program 


Atfwe  Database  Systems 

The  idea  of  incorporating  rules  into  a  database  system  as  integrity 
constraints  and  triggers  came  about  in  with  the  early  CODASYL  systems  in 
the  form  of  ON  conditions.  More  recently,  the  idea  of  combining  rules  and 
data  has  received  much  serious  consideration,  i  he  term  "active  dataoase’ 
has  been  used  frequently  in  referencing  such  databases.  For  example,  rules 
have  been  built  into  POSTGRES  (RoSt87V  there  are  no  differences  between 
constraints  and  triggers;  all  are  implemented  as  a  single  rule  mechanism  In 
addition,  POSTGRES  allows  queries  to  be  stored  in  a  data  field  which  is 
evaluated  whenever  the  field  is  retrieved  In  HiPAC  (DBBC88L  the  concept 
of  Event-Condition- Action  (ECA)  rules  was  proposed  When  an  event 
occurs,  the  condition  is  evaluated;  if  the  condition  is  satisfied,  the  action  is 
executed  It  can  be  shown  that  ECA  rules  can  be  used  to  realize  integrity 
constraints,  alerts,  and  other  facilities.  Rules  have  also  been  included  in  the 
context  of  object-oriented  databases.  In  Starburst  (WiCL9l),  for  example, 
rules  can  be  used  to  enforce  integrity  constraints  and  to  trigger  oorsequt... 
''Tions  In  !rs  {Fish 871,  a  query  can  be  monitored  by  first  defining  it  as  a 
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The  calling  object  resumes  execution  immediately  after  the  reply  is 
received  from  the  called  object 


Communication  between  two  objects  is  said  to  be  asynchronous  if  the 
calling  object  continues  its  execution  after  a  message  is  sent  The  reply 
associated  with  the  message  is  later  picked  up  by  a  recave  operation  These 
can  be  accomplished  with  the  following  method  predicates 


mstndO,  which  is  true  if  a  message  m  is  sent  from  the  object  to 
another  object.  The  message  contains  the  sender,  the  recipient,  a 
timestamp,  the  method  to  invoke,  and  a  set  of  arguments  if 
necessary 

m.receweO  which  is  true  if  a  message  m  is  received  at  the  object 


Based  on  the  above,  an  operation  of  the  form  ca (....)  implements  a 
synchronous  communication;  it  is  equivalent  to  a  send  operation  immediately 
followed  by  a  receive  operation,  f  a  send  operation  is  followed  by  a  receive 
operation  in  a  deterministic  state,  with  some  other  operations  in  between 
them,  then  the  communication  is  more  or  less  synchronous  Get  us  call  it  semi- 
synchronous).  If  a  send  operation  is  followed  by  a  receive  operation  in  an 
indeterministic  state,  then  the  communication  »«  synchronous 


4.  Object -Oriented  Simulation 

As  discussed  in  Section  2.  most  of  the  existing  object-oriented 
simulation  systems  provide  an  object-oriented  user  interface  so  that  a 
simulation  program  can  be  described  in  an  easy  and  friendly  fashion. 
Execution  of  an  object-oriented  simulation  program  can  be  completely 
sequential  or  fully  distributed  as  the  program  specifies-  As  attractive  as  it 
seems,  executing  a  simulation  program  as  a  fully  distributed,  object-oriented 
system  could  be  inefficient  due  to  the  shortage  of  physical  resources  and  the 
overhead  associated  with  process  management.  Motivated  by  this,  our 
approach  is  to  compile  an  object-oriented  simulation  program,  in  which  each 
process  object  is  represented  as  a  production  system,  into  a  (production)  rule 
network.  If  each  process  object  is  treated  as  a  passive  object  then  each  node 
of  the  network  corresponds  to  a  set-oriented  operation.  The  compiled 
network  (or  the  set  of  operations  of  the  network)  is  evaluated  in  parallel 
Changes 
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Figure  1 .  Structure  of  the  simulator. 


to  objects  are  generated  at  the  terminals  of  the  network.  Events  generated  by 
a  process,  if  any,  are  expressed  as  new  production  rules  and  are  compiled 
into  (and  removed  from)  the  network  dynamically.  The  overall  architecture 
of  the  object-oriented  simulator  is  shown  in  Figure  1  When  operational  the 
simulator  executes  a  loop  with  the  following  steps  (see  figure  2): 

Process  Selection 


The  rule  network  is  evaluated.  At  the  terminals  of  the  network,  events 
are  generated.  This  step  basically  selects  those  processes  which  have  one  or 
more  productions  eligible  for  firing  based  on  their  current  states 


Productien  firms 

Fur  h  process  selected,  the  actions  associated  with  each  production 
rule,  which  is  ready  to  Fire,  are  taken.  (Note  that  a  rule,  which  is  ready  to  fire, 
could  be  an  event  created  earlier)  Su~h  '  action  vuuld  be  an  operauon 
which  changes  the  *«uuc  of  a  (passive)  object,  a  communication  operation 
(send  and/or  receive),  or  an  operation  which  produces  an  event  (which  will 


be  executed  m  the  future),  which  is  convert**!  to  the  form  of  prt*iu<l:on  n,ie*> 
and  compiled  into  the  network  by  the  network  builder  If  necessary,  the 
value  of  the  dock  associated  with  the  process  ts  updated  based  on  the 
operation(s)  At  this  point,  if  a  causality  emir  occurs.  n*H«*v>ary  rollbacks  are 
performed  in  the  processes  involved 


Figure  2.  Logic  of  the  simulator 


As  described,  this  approach  merges  a  set  of  process  objects  into  a 
(much)  smaller  set  of  operation  processes  Simulation  can  be  performed 
synchronously  or  asynchronously.  Asynchronous  simulation  is  mure 
complicated  as  different  versions  of  the  same  object  must  be  considered  in 
making  a  decision.  This  will  be  reported  in  a  separate  paper  L.  tU 
synchronous  mode,  a  global  clock  is  employed  so  that  every  process  objec  t  is 
synchronised  with  respect  to  the  global  dock.  The  network  can  be  evaluated 
continuously  every  clock  cycle  or  discretely  according  to  the  events 
produced.  It  requires,  however,  that  the  states  of  each  object  be  recorded  and 
rod  barks  be  performed  whenever  causality  errors  are  detected 

Since  the  object  model  allows  objects  to  be  shared  among  different 
processes  (although  they  are  accessed  through  messages),  serial  liability 
[BeHG87]  must  be  maintained  all  the  time  This  means  that  the  effects 
created  by  multiple  processes  which  are  executed  concurrently  should  be  the 
same  as  those  created  by  a  (any)  serial  schedule  among  the  processes  To 
assure  this,  the  design  employs  the  twophase  locking  protocol  which 
requires  all  objects  accessed  by  a  process  be  locked  before  accessed,  that  al] 
locks  be  acquired  before  any  unlock,  and  that  all  objects  be  unlocked  before 
the  process  terminates.  Clearly,  two- phase  locking  cannot  be  implemented  at 
the  method  level,  because  two  consecutive  method  calls  can  violate  the  two 
phase  requirement  Consequently,  it  is  required  that  each  method  lock  any 
object  it  may  access  but  not  to  subsequently  unlock  it  The  list  of  locked 
objects  should  be  returned  to  the  calling  process  so  that  it  can  unlock  the 
locked  objects  before  terminating. 


5.  Rule  Processing 

In  general,  the  processing  of  production  rules  or  integrity  constraints 
can  create  serious  performance  bottlenecks  when  a  large  volume  of  facts  and 
rules  are  integrated.  Since  multiple  instances  of  the  same  class  share  the 
same  copy  of  production  rules,  it  is  useful  to  compile  a  set  of  rules  into  one 
system  in  which  some  set-oriented  operations  can  be  employed  to  process  the 
data  (treated  as  sets)  collectively.  Furthermore,  given  a  set  of  rules,  it  is 
fruitful  to  merge  those  expressions  that  are  common  to  more  than  one  rule  so 
that  duplicated  effort  can  be  avoided.  A  network  approach  is  taken  for  this 
purpose  This  approach  is  similar  to  the  RETE  algorithm 
((ACAR86)(Forg82|).  but  is  more  general  in  the  treatment  of  logical  formulas 
Although  integrity  constraints  and  production  rules  are  treated  slightly 
differently,  both  are  processed  based  on  a  network  that  is  compiled  from  a  set 
of  logical  formulas. 

Processing  Integrity  Constraints 

Given  a  set  of  constraints  [ft  -♦  .  .  fn  -*  rn^  outset,  each 

constraint  /,  r,  is  converted  into  the  form  ft  a  rr  (i.e  ,  the  negation  of  the 
original  rule).  All  the  converted  rules  are  subsequently  compiled  into  a 
network  (see  below)  in  which  each  rule  corresponds  to  a  ten-run*1 
^♦nm  the  network,  and  there  i«  no  violation  n*  ,hr*  nd*'  u  no  can 

"flow"  out  from  that  terminal 
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where  Cg  U  the  Him  of  the  cardinalities  of  the  base  relations  which  are 
initially  read  (or  joins  or  selections.  Since  MCFs  are  counted  only  once,  the 
sharing  erf  information  in  the  relational  network  are  reflected  in  the  above 
cost  function.  Once  a  network  is  built,  it  can  be  evaluated  incremen¬ 

tally.  Specifically,  ee'+r  operation  is  evaluated  once  based  on  the  initial  state 
erf  the  system.  In  the  meantime,  for  each  operation,  the  results  are  stored. 
Subsequently,  as  the  state  of  the  system  is  changed,  only  those  rules  which 
are  affected  by  a  changed  fact  need  to  be  evaluated  during  each  itmtian. 
When  an  update  of  the  database  is  made,  operations  are  performed  from  the 
bottom  of  the  network,  first,  the  updated  fact  is  matched  against  the  MGPs. 
Only  those  MGPs  having  the  same  head  as  the  updated  fact  and  whose 
arguments  can  be  unified  by  the  arguments  of  the  updated  (act  are  activated. 
After  the  appropriate  MGPs  are  activated,  the  operations  connected  to  the 
activated  MGPa  are  activated  For  each  activated  operation  node,  the  content 
of  the  stored  result  la  changed  according  to  the  changefs)  in  its  input 
relaticna.  If  die  update  is  an  addition  of  a  new  fact,  a  new  tuple  of  values 
may  be  added  after  a  select  operation.  If  the  update  is  a  deletion,  the  tuple 
corresponding  to  the  deleted  fact  may  be  deleted  from  the  result  stored  in  die 
operation  node  after  a  select  operation  Similarly,  tuples  may  be  deleted 
from  the  result  of  a  join  operation  if  some  tuples  are  deleted  from  its  inputs; 
and  tuples  may  be  added  Into  the  result  of  a  join  operation  if  some  tuples  are 
added  into  its  inputs.  A  modification  of  an  existing  fact  can  be  handled  by 
first  deleting  the  old  fact  followed  by  adding  Che  new  fact.  After  all  the 
related  operations  to  an  update  are  performed,  the  final  changes  obtained  at 
each  terminal  node  are  applied  to  the  associated  actfon(a). 
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Figure  3.  A  rule  network. 


selected  to  implement  r  with  the  current  network  1  ne  following  procedure 
can  be  followed  to  identify  ail  the  possible  covers  for  r 


Identifying  Cavers  for  A  New  Rule 

mput  :  N  and  r  as  described  in  the  above 

output:  C,  the  set  of  all  the  covers  for  r  with  respect  to  N. 

Step  J 

Let  C  -*,7  -{!«,)  I  ",  «  Nn  | 

Step  2 

Rnd  u  and  v  of  T  where  /  <u  )  a  /  {v  )  is  more  general  than  a  sub- 
formula  of  r  and  1)  is  not  in  T  or  C,  If  at  this  point  u  a  o  »  r  ,  fcu.  v  I  is 
a  cover  for  r  ;  therefore  C  -C  o{(if,  v  I).  Otherwise,  let  7  -7  o{|u.  p|J 
Repeat  this  step  until  no  more  of  u  and  v  pairs  can  be  found. 

O 


7.  Parallel  Evaluation  of  a  Rale  Network 

The  following  approaches  can  be  applied  in  order  to  evaluate  a  rule 
network,  depending  on  how  logical  objects  are  packed  into  physical  objects 

Oass-Leod  Parallelism 

In  this  approach,  each  node  of  the  network  Is  implemented  as  a 
physical  object,  where  each  terminal  node  is  a  class  object  and  each  internal 
node  is  an  operation  object.  The  network  is  evaluated  as  an  active  network 
which  operates  In  a  pipelined  fashion.  Specifically,  each  operation  object  re¬ 
trieves  inputs  from  Its  Input  object(a)  and  produces  the  outputs,  which  are 
available  to  the  operation  objectis)  at  the  next  higher  level.  Unlike  operation 
nodes,  each  class  object  functions  as  a  data  store  from  which  data  can  be 
retrieved  by  operation  objects. 


Stt-Lad  PtmlMim 

In  this  approach,  each  terminal  node  is  implemented  as  a  set  of  objects, 
in  which  each  object  corresponds  to  a  subset  of  a  class.  The  network  is 
transformed  Into  an  equivalent  network  in  which  each  terminal  node 
corresponds  to  a  subclass  object.  The  transformation  can  be  done  in  a 
straight-forward. fashion  based  on  the  following  principles: 

1.  (R»RjuR2)e.(S»SjcyS2)=sR  Ixl  S  -  (Rj  Ixl  Sj  IwiR] 

Ixl  S2  )<J(R2  IxISj  )u(R2  IxIS^  > 

2.  (R  -  Rj  \jR2  )  =»  srlectf  (R  )  -  selortf  (Rj )  u/sdectp  (R2  > 

Clearly,  this  approach  can  achieve  a  higher  degree  of  parallelism, 
however  it  is  more  complicated  to  Implement  In  addition,  Ihe  number  of 
operator  objects  can  grow  exponentially  as  each  class  is  split  into  smaller  and 
smaller  subsets. 


t  Incremental  Network  Compilation 

Let  N  be  a  rule  network  corresponding  to  the  set  of  production  rules  R 
«  |rj,...,fj|  and  letp  -  pj  a  p2  a  ...  a  pn  be  Ihe  condition  part  of  a  new 
production  rule  r  .  Also  let  N„  |n;,  ,ns)  be  the  set  of  nodes  of  N  and  the 
function,  f.  map  each  n„  1  £  i  £  s,  to  its  corresponding  conjunctive  formula 
A  simulator  requires  that  N  be  revised  with  minimal  effort  to  a  new  network 
N  which  corresponds  to  the  augmented  rule  set  R  -  |rj,....rj/  ). 

Instead  of  recompiling  the  augmented  rule  set  from  scratch,  if  is 
desirable  to  incrementally  connect  the  new  rule  Into  the  existing  network.  To 
realize  this,  a  cooer  of  the  new  rule,  r,  with  respect  to  N  is  defined  to  be  any 
subset  of  1  from  which  r  can  be  derived.  The  cast  of  a  cover  is 

defined  to  be  the  sum  of  the  costs  associated  with  the  nodes  and  the  ares 
which  have  to  be  added  into  the  network  so  that  t  can  be  computed  from  the 
nodes  of  Ihe  cover.  Among  all  the  covers,  the  one  with  the  smallest  cost  is 


A  Primitive  Experiment  Results 

Some  primitive  experiments  have  been  performed  on  an  ENCORE 
parallel  computer  (with  four  processors)  to  compare  the  performance  of 
different  approaches  for  a  simple  scenario.  The  scenario  consists  of  a  number 
of  battle  divisions  comprising  two  sides,  blue  and  red.  The  divisions  are 
initially  located  on  the  boundaries  of  a  battlefield  which  is  modeled  by  a 
square  of  grid  tiles.  The  scenario  is  set  up  so  that  all  Ihe  red  divisions  are 
distributed  on  the  east  border  oi  the  battlefield  and  the  blue  divisions  are 
distributed  on  the  west  side.  Once  initialed,  the  blue  divisions  march  to  the 
west  and  the  red  divisions  march  to  the  east.  Throughout  the  simulation 
each  division  is  characterized  by  its  strength,  speed,  direction  of  movement, 
and  its  location.  When  two  divisions  of  opposite  tides  encounter  each  other, 
the  strength  of  the  weaker  is  reduced  to  zero,  and  the  strength  of  the  stronger 
is  reduced  by  that  of  the  weaker.  On  the  other  hand,  when  two  divisions  of 
the  same  side  meet  with  each  other,  they  are  merged  into  a  larger  division 
In  any  case,  the  number  of  divisions  in  each  gnd  tile  cannot  exceed  two 

The  scenario  was  simulated  using  four  approaches:  sequential  (S  ), 
parallel  synchronous  ( PS  ),  asynchronous  (A  ),  and  the  network  approach  (N  !  as 
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